Agent-Based Modeling for Systems of Systems

Agent-based modeling is an important tool for the engineering of systems of systems. This paper briefly reviews the historical development of agent-based modeling and system of systems concepts, compares agent-based modeling to other approaches, and describes the Purdue Discrete Agent Framework for agent-based modeling. An application of the Discrete Agent Framework to a system of systems is described and illustrates the ability of agent-based modeling to capture non-intuitive behaviors that may arise due to the complex dynamics that occur in interconnected systems of agents that follow a behavioral set of rules. The paper concludes by looking back to the past to understand the potential for applying agent-based modeling to support the ongoing engineering and operations of an evolving system of systems. Introduction Agent-based modeling employs a collection of autonomous decision-making entities called agents imbued with rules of behavior that direct their interaction with each other and their environment. Agent functionality is quite flexible, with behavior types ranging from simply reactive (change state or take action based on fixed rules) to learning and adaptive (change state or take action after updating internal logic schema via learning). Agent-based modeling is an important tool for the engineering of systems of systems. Using this approach, systems engineers can investigate alternative architectures and gain an understanding of the impact of the behaviors of individual systems on emergent behaviors. The purpose of this paper is to provide an understanding of the concept of systems of systems and the application of agent-based modeling to systems of systems. We review important literature on systems of systems and agent-based modeling, and we compare agent-based modeling to other methods for analyzing systems of systems. We describe a simulation tool that provides a capability for agent-based modeling and simulation and describe the application of this tool to a system of systems. Finally, we describe a successful application of the concepts of agent-based modeling to a system of systems that has been in operation for multiple decades. Brief Review of the Literature on Systems of Systems and Agent-Based Modeling A working paper from the Santa Fe Institute by Marimon, McGrattan, and Sargent (1989) used an agent-based simulation to analyze money as a medium of exchange in an economic system. This paper may be the earliest publication that describes simulating agents rather than performing closed-form mathematical analysis of a system of agents. About the same time, Eisner, Marciniak, and McMillan (1991) used the term “system of systems” in a paper that described the need for a new discipline that could provide overall management control over independently acquired systems that have different time phasing and interdependent coupling and that tend to be uni-functional individually while the systems of systems is multi-functional. In addition, optimizing the individual systems does not guarantee optimization of the overall systems of systems and the combined operation of the constituent systems is necessary to achieve an overall mission objective. They described specialized computer-based tools for developing a concept of operations, managing an integrated acquisition master schedule, analyzing network performance, prototyping user interfaces, tracking configuration baselines for constituent systems, and interface compatibility modeling and analysis. They developed their approach to manage the acquisition of a threat warning and attack assessment system of systems, and they foresaw the need to develop system-of-systems methods applicable to air transportation, digital communications, and strategic missile defense. They refer to “the extensive use of automated models and simulations in order to predict performance” (532). Eisner, McMillan, Marciniak, and Pragluski (1992) described the Rapid Computer-Aided System of Systems Engineering (RCASSE) environment that they structured using ten elements: mission engineering, baseline architecting, performance assessment, specialty engineering, interface compatibility evaluation, software evaluation, risk management, scheduling, pre-planned product improvement, and life cycle cost evaluation. They point out that “The advance of computer networks has created a complexity that certainly did not exist with individual, non-automatically interfaced systems” (268), and they indicate that under the RCASSE process results of performance assessment from automated modeling and simulation tools feed back to the preceding step to modify, as necessary, the baseline architecture. Maier (1996) described five principal characteristics of systems of systems that distinguish them from monolithic systems: operational independence, managerial independence, evolutionary development, emergent behavior, and geographic distribution. Maier (1998) identified the first two of the five characteristics as the necessary and sufficient properties for a collections of systems to be regarded as a systems of system. He also concluded that for systems of systems, “it is apparent that the architecture of each is defined through communications” (280) and that “Collaborative and virtual systems-of-systems will also become more common with the ubiquity of smart systems independently operated and managed” (283). Clymer (1997) defines an agent-based system architecture as a set of agents, the activities or functions each agent performs, the interactions of each agent with the environment, and how each agent communicates with other agents. When we refer to the term “architecture” in this paper, we are using Clymer’s definition. Clymer’s methodology was intended to be applied by systems engineers to realize complex adaptive systems, which “are comprised of a collection of agents where each agent in the system communicates data, knowledge (rules and facts), and mission goals with one or more other agents” (1). DeLaurentis (2005) expanded Maier’s five characteristics of a system of systems to include networks, heterogeneity, and trans-domain. By trans-domain, DeLaurentis means that effective study of systems of systems requires unifying knowledge across several fields of study: engineering, economics, policy, and operations. He evaluated the four system views of Rouse (2003) (hierarchical mapping, state equations, nonlinear mechanism, and autonomous agents) for applicability to systems of systems and concluded that the modeling of a system of systems as autonomous agents is well suited for capturing the emergent behavior that derives from complex interactions of the other six characteristics of systems of systems. He also described how object-oriented methods could be effective and efficient for implementing agent-based models. Hsu and Butterfield (2007) defined four principles of emergence and proposed that agent-based modeling to measure the existence, type, of level of emergent behavior of systems of systems and the initiation mechanisms for the emergent behavior. The recent increase in availability of software packages for agent-based simulation and the increased understanding that agent-based modeling is well suited for systems of systems has resulted in many applications of agent-based modeling to systems of systems. Kilicay-Ergin and Dagli (2008) describe the use of AnyLogic agent-based simulation software to model the behavior of alternative system-of-system architectures for financial markets. Hsu, Price, Clymer, Garcia, and Gonzalez (2009) describe using OpEMCSS software to simulate the behavior of a system of systems that they call the World Model. Giachetti, Marcelli, Cifuentes, and Rojas describe a simulation that uses Java-based CybelePro software to model the performance of a human-robot team as an agent-based system of systems. Comparison of Agent Based Modeling to Other Approaches Various methods and tools for simulation of complex processes exist; however, they primarily fall into the main categories of equation-based system dynamics, discrete event simulation, and agent-based modeling. The following section provides comparative discussion on the differences between agent-based models and the other methods. System dynamics is defined as the “the study of information-feedback characteristics of industrial activity to show how organizational structure, amplification (in policies), and time delays (in decisions and actions) interact to influence the success of the enterprise” (Forrester 1958, 38). The system dynamics methodology uses mathematical equations to represent feedback loops of stock flows throughout a process network. An important assumption for system dynamics is that individual stock items are indistinguishable and that the resulting feedback loop generated is an adequate representation of each individual stock flow’s aggregate behavior. Typically, system dynamics represents an aggregate level of performance as continuous differential equations. Analysts employ these models support strategic-level decision-making and to develop an overarching view of long-term trends in the dynamics of an enterprise. System dynamics has been widely used across a range of applications that range from socio-economic to engineering systems, and aims to reduce complex behaviors to their most aggregate forms assuming that adequate, structured representations of the behaviors exist. Agent-based modeling is a natural choice for analyzing systems of systems because (1) it can capture emergent phenomena, (2) it provides a natural description of the system, and (3) it is flexible (Bonabeau 2002). Schieritz and Milling (2003) and Borshchev and Filippov (2004) provide a comprehensive comparison of simulations that use systems dynamics versus simulations that use agent-based modeling. System dynamics focuses on a top-down, aggregate modeling that typically uses continuous-form representations (feedback loops) of system processes; agent-based models are based on discrete agent-specific logic rules that take a bottom-up approach to simulation. Agent-based modeling provides a means of connecting micro-level behaviors to the macro level of a system whereas systems dynamics link system structures to system behavior (Schieritz and Milling 2003, Borshchev and Filippov 2004). The main difference is in the ability of agent-based models to capture emergent behaviors. The agent-based setting allows flexible interactions between individual agents, which results in non-intuitive dynamic modes being generated. System dynamics reduces the possibility of exploring emergent phenomena due to the natural filtering of these modes that occurs through enforcement of aggregate equations over populations of individuals within the system, and the establishment of a rigid flow structure. Discrete Event Simulation is a method used to model real world processes as a series of interconnected discrete events that are functional processes. These processes are typically at the lowto mid-level state of abstraction in the hierarchy of interconnected systems and do not consider performance characteristics of the individual elements that execute these processes themselves. The focus of a process-centric simulation here is naturally well suited to applications where processes are the critical aspect of analysis such as in healthcare (e.g., patient flow), manufacturing (e.g., production floor processes layout), and logistics (e.g., distribution processes at a hub). As with the system dynamics approach, the discrete event simulation approach is a top-down approach that models aggregate behaviors of processes. A more recent method of discrete event simulation uses Petri nets, a mathematical and graphical method that employs triggers, tokens, and transitions to model interactions between entities (Murata 1989). Further developments of Petri nets have resulted in colored Petri nets where colored tokens represent the flow of different packets of information along the feasible pathways of the Petri network. Much literature has proliferated on Petri nets due to their simplicity and computational ease of use. Additionally, they bear some useful properties in comparison to other methods. For example, when contrasted to Markov chains, Petri nets do not require an increase in the number of states (and consequently state variables) with an increase in the number of tokens used in the model. This preserves the computational complexity of the underlying simulation and scales well to larger problems. In addition, Petri nets easily handle serial and concurrent execution of process events. While discrete methods use a powerful and intuitive representation of processes in a system, they are mainly intended to model and represent finite interactions where the underlying structure of the process is already known. They share the focus with systems dynamics of modeling top-down characteristics of a system and assume pre-defined structures and aggregations of macro behaviors. In contrast, agent-based models are able to more generally represent individual entities that drive the discrete events and allow for possible emergent behaviors that are not otherwise apparent from the aggregated discrete dynamics of a system. DAF Approach to Agent-Based Modeling Purdue University developed the Discrete Agent Framework (DAF) for agent-based modeling in 2010 to enable easy application in multiple domains. Developed in object-oriented MATLAB, this engine provides the foundation to build agent-based simulation models to explore various architecture configurations for systems of systems and evaluate their performance. DAF also enables coordinated development, verification, and validation of the system of systems architecture through selective failure simulation. DAF allows the modeling effort to focus on the systems of systems itself and not the logistical “dirty work” of getting a runnable simulation from a blank slate. The first major application of DAF was through a sponsored research initiative of the Missile Defense Agency of the US Department of Defense. The purpose of this research was to examine and model a Ballistic Missile Defense System (BMDS) as a system of systems by simulating it as a collection of functions (executed by agents) that could be distributed in a myriad of ways. For example, a ground-up development effort to model a BMDS would require coding not only agent behavior, but also routing of communication between agents. Development in DAF reduces the routing into a single command, allowing the modeler to focus on agent behavior. The engine provides the means and a head start to addressing top-level objectives, developing mathematical models and agent behavior algorithms, and defining the architectural design space. DAF views a system of systems architecture as a collection of agents that are connected by communication links. In practice, each agent in DAF is an in-code application of a formal model developed from research and of communication links that emulate real or proposed communication standards (Chow, Braun, and Fry 2012). This approach allows a DAF user to follow Maier’s communication-centric architecting approach (Maier 1998) by using the different possibilities of linking these agents as a means to distinguish one architecture from another. DAF can used to generate and evaluate a wide variety of architectures by defining the functional capabilities and behaviors of agents and the communication links between the agents. A representative implementation of DAF involves generating architecture alternatives, then simulating them to identify the configuration that provided the best balance of efficiency and reliability. There are certainly many agent-based modeling packages available and each provides many combinations of capability, ease-of-use, and availability. For example, SWARM is an open source Objective C/Java-driven simulation system for modeling complex systems through discrete event simulation. NetLogo is another Java-based package that provides multi-platform complex system simulation, but also comes with a large database of sample models and implementations. Initial development work on the SWARM system indicates that an object-oriented development environment is ideal for building agent-based simulations. The primary differentiator between DAF and other packages is that DAF is MATLAB-based, and as a result, any DAF application can utilize the many mathematical, statistical, and visualization tools built into MATLAB or the many supported and third-party toolboxes associated with MATLAB. Additionally, the widespread use of MATLAB in research and industry should reduce the time required to learn how to use DAF and the time to apply it to a particular project. Results of Applying DAF to A Littoral Operations Scenario In order to illustrate the capabilities of agent-based modeling using DAF, we developed a simulation model to capture the performance of a Littoral Combat Ship (LCS) squadron in a scenario involving multiple threats. The LCS is a frigate-sized, modular platform optimized for operating in the littorals, or coastal areas. The LCS is distinctive for its modular “plug-and-fight” mission packages. Rather than being a standalone, multi-mission ship, the ship’s mission orientation can be changed by changing its mission packages. (O’Rourke 2012). The LCS without any mission packages is referred to as the LCS seaframe. The seaframe forms the core of the LCS and provides basic self-defense capability through sensors, weapons, and speed while the mission packages form the bulk of the war fighting capability of LCS. The seaframe is augmented by mission packages that are focused in one of three mission areas: Surface Warfare (SUW), Anti-Submarine Warfare (ASW), or Mine Counter-Measures (MCM). The SUW mission package adds a MH-60R helicopter armed with Hellfire missiles, the Non-Line of Sight (NLOS) missile system, and a Vertical Take-Off Unmanned Aerial Vehicle (UAV). The SUW mission package provides maritime security and prosecution of small boat threats in littorals. The ASW mission package uses off-board technology to detect, classify, localize, and prosecute threat submarines. The package includes Unmanned Surface Vehicles (USVs), Remote Manned Vehicles (RMVs), and the MH-60R helicopter that both employ a dipping sonar for the detection of sub-surface targets. The ASW LCS does not have an anti-submarine weapon and is dependent on its MH-60R helicopter to deliver anti-submarine weapons. Finally, the MCM mission package is dependent on its helicopter for neutralization of detected mines. The USVs and Remote Mine-hunting Systems (RMS) in the MCM mission package use towed bodies to detect mines. Figure 1 shows a hierarchical view of the system of systems for the littoral operations scenario using a lexicon developed by DeLaurentis and Callaway (2004). The collection of entities at the lowest level of the lexicon (indicated by α in the figure) and their connectivity determines the construct of a β-level collection of LCS Mission Packages, Surface Threats, Sub-Surface Threats, and Merchants. The collection for a littoral region (γ-level) is a collection of β-level entities, and the δ-level is a set of National and International Institutions to which the γ entities belong. DeLaurentis and Callaway (2004) also define four categories (resources, operations, economics, and policy) that are used in Table 1 to describe the breadth of the SoS problem as well as to help guide modeling and simulation of the SoS. Figure 1. The SoS Hierarchy for Littoral Operations Table 1. Description of SoS Categories for Littoral Operations Level Resources Operations Economics Policy α LCS, MH60, UAV, USV, RMV, Boat, Mine, Submarine, Merchant Ship Operating a single resource like LCS, Submarine, etc. Economics of acquisition and operation of single entity during a patrol or voyage Policies regarding single-resource usage: crew make-up, roles, and responsibilities; weapon loadout and firing doctrine, etc. β SUW / ASW / MCM Mission Package, Surface Threats, Sub-surface Threats, Merchant Traffic Mission package operations, threat operations, merchant ship operations Economics of military patrols and merchant voyages Policies regarding LCS package, threat, and merchant operations: communications protocols, engagement tactics, evasion tactics, etc.